Super-resolution image reconstruction is a form of digital image processing that increases the resolvable detail in images. The earliest techniques for super-resolution generated a still image of a scene from a collection of similar lower-resolution images of the same scene. For example, several frames of low-resolution video may be combined using super-resolution techniques to produce a single still image whose resolution is significantly higher than that of any single frame of the original video. Because each low-resolution frame is slightly different and contributes some unique information that is absent from the other frames, the reconstructed still image has more information, i.e., higher resolution, than that of any one of the originals alone. Super-resolution techniques have many applications in diverse areas such as medical imaging, remote sensing, surveillance, still photography, and motion pictures.
The details of how to reconstruct the best high-resolution image from multiple low-resolution images is a complicated problem that has been an active topic of research for many years, and many different techniques have been proposed. One reason the super-resolution reconstruction problem is so challenging is because the reconstruction process is, in mathematical terms, an under-constrained inverse problem. In the mathematical formulation of the problem, the known low-resolution images are represented as resulting from a transformation of the unknown high-resolution image by effects of image warping due to motion, optical blurring, sampling, and noise. When the model is inverted, the original set of low-resolution images does not, in general, determine a single high-resolution image as a unique solution. Moreover, in cases where a unique solution is determined, it is not stable, i.e., small noise perturbations in the images can result in large differences in the super-resolved image. To address these problems, super-resolution techniques require the introduction of additional assumptions (e.g., assumptions about the nature of the noise, blur, or spatial movement present in the original images). Part of the challenge rests in selecting constraints that sufficiently restrict the solution space without an unacceptable increase in the computational complexity. Another challenge is to select constraints that properly restrict the solution space to good high-resolution images for a wide variety of input image data. For example, constraints that are selected to produce optimal results for a restricted class of image data (e.g., images limited to pure translational movement between frames and common space-invariant blur) may produce significantly degraded results for images that deviate even slightly from the restricted class.
The multiframe super-resolution problem was first addressed by a proposed frequency domain approach. Although the frequency domain methods are intuitively simple and computationally cheap, they are extremely sensitive to model errors, limiting their use. Also, by definition, only pure translational motion can be treated with such tools and even small deviations from translational motion significantly degrade performance.
Another popular class of methods solves the problem of resolution enhancement in the spatial domain. Non-iterative spatial domain data fusion approaches have been proposed, and an iterative back-projection method was previously developed. Additionally, a method based on the multichannel sampling theorem has been suggested. Further, a hybrid method, combining the simplicity of maximum likelihood (ML) with proper prior information was suggested.
The spatial domain methods known in the art are generally computationally expensive. A block circulant preconditioner for solving the Tikhonov regularized super-resolution problem has been introduced and formulated, and addressed the calculation of regularization factor for the under-determined case by generalized cross validation. Later, a very fast super-resolution algorithm for pure translational motion and common space invariant blur was developed. Another fast spatial domain method was recently suggested, where LR images are registered with respect to a reference frame defining a nonuniformly spaced high-resolution (HR) grid. Then, an interpolation method called Delaunay triangulation is used for creating a noisy and blurred HR image, which is subsequently deblurred. All of the above methods assumed the additive Gaussian noise model. Furthermore, regularization was either not implemented or it was limited to Tikhonov regularization. Considering outliers, a very successful robust super-resolution method has been described, but lacks the proper mathematical justification. Finally, quantization of noise resulting from video compression and proposed iterative methods have been considered to reduce compression noise effects in the super-resolved outcome.
What is needed is a super-resolution technique that is computationally efficient and produces desired improvements in image quality that are robust to variations in the properties of input image data.